{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Baseline_Feature_Transformation)=\n",
    "# Baseline feature transformation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simulated dataset generated in the previous section is simple. It only contains the essential features that characterize a payment card transaction. These are: a unique identifier for the transaction, the date and time of the transaction, the transaction amount, a unique identifier for the customer, a unique number for the merchant, and a binary variable that labels the transaction as legitimate or fraudulent (0 for legitimate or 1 for fraudulent). Fig. 1 provides the first three rows of the simulated dataset:\n",
    " \n",
    "![alt text](images/tx_table.png)\n",
    "<p style=\"text-align: center;\">\n",
    "Fig. 1. The first three transactions in the simulated dataset used in this chapter.\n",
    "</p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What each row essentially summarizes is that, at 00:00:31, on the 1st of April 2018, a customer with ID 596 made a payment of a value of 57.19 to a merchant with ID 3156, and that the transaction was not fraudulent. Then, at 00:02:10, on the 1st of April 2018, a customer with ID 4961 made a payment of a value of 81.51 to a merchant with ID 3412, and that the transaction was not fraudulent. And so on. The simulated dataset is a long list of such transactions (1.8 million in total). The variable `transaction_ID` is a unique identifier for each transaction.  \n",
    "\n",
    "While conceptually simple for a human, such a set of features is however not appropriate for a machine learning predictive model. Machine learning algorithms typically require *numerical* and *ordered* features. Numerical means that the type of the variable must be an integer or a real number. Ordered means that the order of the values of a variable is meaningful. \n",
    "\n",
    "In this dataset, the only numerical and ordered features are the transaction amount and the fraud label. The date is a Panda timestamp, and therefore not numerical. The identifiers for the transactions, customers, and terminals are numerical but not ordered: it would not make sense to assume for example that the terminal with ID 3548 is 'bigger' or 'larger' than the terminal with ID 1983. Rather, these identifiers represent distinct 'entities', which are referred to as *categorical* features. \n",
    "\n",
    "There is unfortunately no standard procedure to deal with non-numerical or categorical features. The topic is known in the machine learning literature as *feature engineering* or *feature transformation*. In essence, the goal of feature engineering is to design new features that are assumed to be relevant for a predictive problem. The design of these features is usually problem-dependent, and involves domain knowledge.\n",
    "\n",
    "In this section, we will implement three types of feature transformation that are known to be relevant for payment card fraud detection.\n",
    "\n",
    "![encoding](images/encoding_variables.png)\n",
    "\n",
    "The first type of transformation involves the date/time variable, and consists in creating binary features that characterize potentially relevant periods. We will create two such features. The first one will characterize whether a transaction occurs during a weekday or during the weekend. The second will characterize whether a transaction occurs during the day or the night. These features can be useful since it has been observed in real-world datasets that fraudulent patterns differ between weekdays and weekends, and between the day and night.  \n",
    "\n",
    "The second type of transformation involves the customer ID and consists in creating features that characterize the customer spending behaviors. We will follow the RFM (Recency, Frequency, Monetary value) framework proposed in {cite}`VANVLASSELAER201538`, and keep track of the average spending amount and number of transactions for each customer and for three window sizes. This will lead to the creation of six new features.\n",
    "\n",
    "The third type of transformation involves the terminal ID and consists in creating new features that characterize the 'risk' associated with the terminal. The risk will be defined as the average number of frauds that were observed on the terminal for three window sizes. This will lead to the creation of three new features. \n",
    "\n",
    "The table below summarizes the types of transformation that will be performed and the new features that will be created. \n",
    "\n",
    "|Original feature name|Original feature type|Transformation|Number of new features|New feature(s) type|\n",
    "|---|---|---|---|---|\n",
    "|TX\\_DATE\\_TIME | Panda timestamp |0 if transaction during a weekday, 1 if transaction during a weekend. The new feature is called TX_DURING_WEEKEND.|1|Integer (0/1)|\n",
    "|TX\\_DATE\\_TIME | Panda timestamp |0 if transaction between 6am and 0pm, 1 if transaction between 0pm and 6am. The new feature is called TX_DURING_NIGHT.|1|Integer (0/1)|\n",
    "|CUSTOMER\\_ID | Categorical variable |Number of transactions by the customer in the last n day(s), for n in {1,7,30}. The new features are called CUSTOMER_ID_NB_TX_nDAY_WINDOW.|3|Integer|\n",
    "|CUSTOMER\\_ID | Categorical variable |Average spending amount in the last n day(s), for n in {1,7,30}. The new features are called CUSTOMER_ID_AVG_AMOUNT_nDAY_WINDOW.|3|Real|\n",
    "|TERMINAL\\_ID | Categorical variable |Number of transactions on the terminal in the last n+d day(s), for n in {1,7,30} and d=7. The parameter d is called delay and will be discussed later in this notebook. The new features are called TERMINAL_ID_NB_TX_nDAY_WINDOW.|3|Integer|\n",
    "|TERMINAL\\_ID | Categorical variable |Average number of frauds on the terminal in the last n+d day(s), for n in {1,7,30} and d=7. The parameter d is called delay and will be discussed later in this notebook. The new features are called TERMINAL_ID_RISK_nDAY_WINDOW.|3|Real|\n",
    "\n",
    "The following sections provide the implementation for each of these three transformations. After the transformations, a set of 14 new features will be created. Note that some of the features are the result of aggregation functions over the values of other features or conditions (same customer, given time window). These features are often referred to as *aggregated features*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": [
     "hide-cell"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
      "                                 Dload  Upload   Total   Spent    Left  Speed\n",
      "100 31567  100 31567    0     0   135k      0 --:--:-- --:--:-- --:--:--  135k\n",
      "Cloning into 'simulated-data-raw'...\n",
      "remote: Enumerating objects: 189, done.\u001b[K\n",
      "remote: Counting objects: 100% (189/189), done.\u001b[K\n",
      "remote: Compressing objects: 100% (187/187), done.\u001b[K\n",
      "remote: Total 189 (delta 0), reused 186 (delta 0), pack-reused 0\u001b[K\n",
      "Receiving objects: 100% (189/189), 28.04 MiB | 3.13 MiB/s, done.\n"
     ]
    }
   ],
   "source": [
    "# Initialization: Load shared functions and simulated data \n",
    "\n",
    "# Load shared functions\n",
    "!curl -O https://raw.githubusercontent.com/Fraud-Detection-Handbook/fraud-detection-handbook/main/Chapter_References/shared_functions.py\n",
    "%run shared_functions.py\n",
    "\n",
    "# Get simulated data from Github repository\n",
    "if not os.path.exists(\"simulated-data-raw\"):\n",
    "    !git clone https://github.com/Fraud-Detection-Handbook/simulated-data-raw\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading of dataset\n",
    "\n",
    "Let us first load the transaction data simulated in the previous notebook. We will load the transaction files from April to September. Files can be loaded using the `read_from_files` function in the [shared functions](shared_functions) notebook. The function was put in this notebook since it will be used frequently throughout this book.\n",
    "\n",
    "The function takes as input the folder where the data files are located, and the dates that define the period to load (between `BEGIN_DATE` and `END_DATE`). It returns a DataFrame of transactions. The transactions are sorted by chronological order. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Load  files\n",
      "CPU times: user 3.1 s, sys: 696 ms, total: 3.79 s\n",
      "Wall time: 4.13 s\n",
      "1754155 transactions loaded, containing 14681 fraudulent transactions\n"
     ]
    }
   ],
   "source": [
    "DIR_INPUT='./simulated-data-raw/data/' \n",
    "\n",
    "BEGIN_DATE = \"2018-04-01\"\n",
    "END_DATE = \"2018-09-30\"\n",
    "\n",
    "print(\"Load  files\")\n",
    "%time transactions_df=read_from_files(DIR_INPUT, BEGIN_DATE, END_DATE)\n",
    "print(\"{0} transactions loaded, containing {1} fraudulent transactions\".format(len(transactions_df),transactions_df.TX_FRAUD.sum()))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2018-04-01 00:00:31</td>\n",
       "      <td>596</td>\n",
       "      <td>3156</td>\n",
       "      <td>57.16</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2018-04-01 00:02:10</td>\n",
       "      <td>4961</td>\n",
       "      <td>3412</td>\n",
       "      <td>81.51</td>\n",
       "      <td>130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2018-04-01 00:07:56</td>\n",
       "      <td>2</td>\n",
       "      <td>1365</td>\n",
       "      <td>146.00</td>\n",
       "      <td>476</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2018-04-01 00:09:29</td>\n",
       "      <td>4128</td>\n",
       "      <td>8737</td>\n",
       "      <td>64.49</td>\n",
       "      <td>569</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>2018-04-01 00:10:34</td>\n",
       "      <td>927</td>\n",
       "      <td>9906</td>\n",
       "      <td>50.99</td>\n",
       "      <td>634</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  TX_AMOUNT  \\\n",
       "0               0 2018-04-01 00:00:31          596         3156      57.16   \n",
       "1               1 2018-04-01 00:02:10         4961         3412      81.51   \n",
       "2               2 2018-04-01 00:07:56            2         1365     146.00   \n",
       "3               3 2018-04-01 00:09:29         4128         8737      64.49   \n",
       "4               4 2018-04-01 00:10:34          927         9906      50.99   \n",
       "\n",
       "   TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  TX_FRAUD_SCENARIO  \n",
       "0               31             0         0                  0  \n",
       "1              130             0         0                  0  \n",
       "2              476             0         0                  0  \n",
       "3              569             0         0                  0  \n",
       "4              634             0         0                  0  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Date and time transformations\n",
    "\n",
    "We will create two new binary features from the transaction dates and times:\n",
    "\n",
    "* The first will characterize whether a transaction occurs during a weekday (value 0) or a weekend (1), and will be called `TX_DURING_WEEKEND`\n",
    "* The second will characterize whether a transaction occurs during the day or during the day (0) or during the night (1). The night is defined as hours that are between 0pm and 6am. It will be called `TX_DURING_NIGHT`. \n",
    "\n",
    "For the `TX_DURING_WEEKEND` feature, we define a function `is_weekend` that takes as input a Panda timestamp, and returns 1 if the date is during a weekend, or 0 otherwise. The timestamp object conveniently provides the `weekday` function to help in computing this value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_weekend(tx_datetime):\n",
    "    \n",
    "    # Transform date into weekday (0 is Monday, 6 is Sunday)\n",
    "    weekday = tx_datetime.weekday()\n",
    "    # Binary value: 0 if weekday, 1 if weekend\n",
    "    is_weekend = weekday>=5\n",
    "    \n",
    "    return int(is_weekend)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is then straghtforward to compute this feature for all transactions using the Panda `apply` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.54 s, sys: 247 ms, total: 7.79 s\n",
      "Wall time: 7.94 s\n"
     ]
    }
   ],
   "source": [
    "%time transactions_df['TX_DURING_WEEKEND']=transactions_df.TX_DATETIME.apply(is_weekend)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We follow the same logic to implement the `TX_DURING_NIGHT` feature. First, a function `is_night` that takes as input a Panda timestamp, and returns 1 if the time is during the night, or 0 otherwise. The timestamp object conveniently provides the hour property to help in computing this value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def is_night(tx_datetime):\n",
    "    \n",
    "    # Get the hour of the transaction\n",
    "    tx_hour = tx_datetime.hour\n",
    "    # Binary value: 1 if hour less than 6, and 0 otherwise\n",
    "    is_night = tx_hour<=6\n",
    "    \n",
    "    return int(is_night)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.09 s, sys: 221 ms, total: 7.31 s\n",
      "Wall time: 7.47 s\n"
     ]
    }
   ],
   "source": [
    "%time transactions_df['TX_DURING_NIGHT']=transactions_df.TX_DATETIME.apply(is_night)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us check that these features where correctly computed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>TX_DURING_NIGHT</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>288062</th>\n",
       "      <td>288062</td>\n",
       "      <td>2018-05-01 00:01:21</td>\n",
       "      <td>3546</td>\n",
       "      <td>2944</td>\n",
       "      <td>18.71</td>\n",
       "      <td>2592081</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288063</th>\n",
       "      <td>288063</td>\n",
       "      <td>2018-05-01 00:01:48</td>\n",
       "      <td>206</td>\n",
       "      <td>3521</td>\n",
       "      <td>18.60</td>\n",
       "      <td>2592108</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288064</th>\n",
       "      <td>288064</td>\n",
       "      <td>2018-05-01 00:02:22</td>\n",
       "      <td>2610</td>\n",
       "      <td>4470</td>\n",
       "      <td>66.67</td>\n",
       "      <td>2592142</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288065</th>\n",
       "      <td>288065</td>\n",
       "      <td>2018-05-01 00:03:15</td>\n",
       "      <td>4578</td>\n",
       "      <td>1520</td>\n",
       "      <td>79.41</td>\n",
       "      <td>2592195</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>288066</th>\n",
       "      <td>288066</td>\n",
       "      <td>2018-05-01 00:03:51</td>\n",
       "      <td>1246</td>\n",
       "      <td>7809</td>\n",
       "      <td>52.08</td>\n",
       "      <td>2592231</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754150</th>\n",
       "      <td>1754150</td>\n",
       "      <td>2018-09-30 23:56:36</td>\n",
       "      <td>161</td>\n",
       "      <td>655</td>\n",
       "      <td>54.24</td>\n",
       "      <td>15810996</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754151</th>\n",
       "      <td>1754151</td>\n",
       "      <td>2018-09-30 23:57:38</td>\n",
       "      <td>4342</td>\n",
       "      <td>6181</td>\n",
       "      <td>1.23</td>\n",
       "      <td>15811058</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754152</th>\n",
       "      <td>1754152</td>\n",
       "      <td>2018-09-30 23:58:21</td>\n",
       "      <td>618</td>\n",
       "      <td>1502</td>\n",
       "      <td>6.62</td>\n",
       "      <td>15811101</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754153</th>\n",
       "      <td>1754153</td>\n",
       "      <td>2018-09-30 23:59:52</td>\n",
       "      <td>4056</td>\n",
       "      <td>3067</td>\n",
       "      <td>55.40</td>\n",
       "      <td>15811192</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754154</th>\n",
       "      <td>1754154</td>\n",
       "      <td>2018-09-30 23:59:57</td>\n",
       "      <td>3542</td>\n",
       "      <td>9849</td>\n",
       "      <td>23.59</td>\n",
       "      <td>15811197</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1466093 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "288062           288062 2018-05-01 00:01:21         3546         2944   \n",
       "288063           288063 2018-05-01 00:01:48          206         3521   \n",
       "288064           288064 2018-05-01 00:02:22         2610         4470   \n",
       "288065           288065 2018-05-01 00:03:15         4578         1520   \n",
       "288066           288066 2018-05-01 00:03:51         1246         7809   \n",
       "...                 ...                 ...          ...          ...   \n",
       "1754150         1754150 2018-09-30 23:56:36          161          655   \n",
       "1754151         1754151 2018-09-30 23:57:38         4342         6181   \n",
       "1754152         1754152 2018-09-30 23:58:21          618         1502   \n",
       "1754153         1754153 2018-09-30 23:59:52         4056         3067   \n",
       "1754154         1754154 2018-09-30 23:59:57         3542         9849   \n",
       "\n",
       "         TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "288062       18.71          2592081            30         0   \n",
       "288063       18.60          2592108            30         0   \n",
       "288064       66.67          2592142            30         0   \n",
       "288065       79.41          2592195            30         0   \n",
       "288066       52.08          2592231            30         0   \n",
       "...            ...              ...           ...       ...   \n",
       "1754150      54.24         15810996           182         0   \n",
       "1754151       1.23         15811058           182         0   \n",
       "1754152       6.62         15811101           182         0   \n",
       "1754153      55.40         15811192           182         0   \n",
       "1754154      23.59         15811197           182         0   \n",
       "\n",
       "         TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  TX_DURING_NIGHT  \n",
       "288062                   0                  0                1  \n",
       "288063                   0                  0                1  \n",
       "288064                   0                  0                1  \n",
       "288065                   0                  0                1  \n",
       "288066                   0                  0                1  \n",
       "...                    ...                ...              ...  \n",
       "1754150                  0                  1                0  \n",
       "1754151                  0                  1                0  \n",
       "1754152                  0                  1                0  \n",
       "1754153                  0                  1                0  \n",
       "1754154                  0                  1                0  \n",
       "\n",
       "[1466093 rows x 11 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_df[transactions_df.TX_TIME_DAYS>=30]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 2018-05-01 was a Monday, and the 2018-09-30 a Sunday. These dates are correctly flagged as weekday, and weekend, respectively. The day and night feature is also correctly set for the first transactions, that happen closely after 0 pm, and the last transactions that happen closely before 0 pm. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Customer ID transformations\n",
    "\n",
    "Let us now proceed with customer ID transformations. We will take inspiration from the RFM (Recency, Frequency, Monetary value) framework proposed in {cite}`VANVLASSELAER201538`, and compute two of these features over three time windows. The first feature will be the number of transactions that occur within a time window (Frequency). The second will be the average amount spent in these transactions (Monetary value). The time windows will be set to one, seven, and thirty days. This will generate six new features. Note that these time windows could later be optimized along with the models using a model selection procedure ([Chapter 5](Model_Selection)). \n",
    "\n",
    "Let us implement these transformations by writing a `get_customer_spending_behaviour_features` function. The function takes as inputs the set of transactions for a customer and a set of window sizes. It returns a DataFrame with the six new features. Our implementation relies on the Panda `rolling` function, which makes easy the computation of aggregates over a time window.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": [
     "hide-cell"
    ]
   },
   "outputs": [],
   "source": [
    "def get_customer_spending_behaviour_features(customer_transactions, windows_size_in_days=[1,7,30]):\n",
    "    \n",
    "    # Let us first order transactions chronologically\n",
    "    customer_transactions=customer_transactions.sort_values('TX_DATETIME')\n",
    "    \n",
    "    # The transaction date and time is set as the index, which will allow the use of the rolling function \n",
    "    customer_transactions.index=customer_transactions.TX_DATETIME\n",
    "    \n",
    "    # For each window size\n",
    "    for window_size in windows_size_in_days:\n",
    "        \n",
    "        # Compute the sum of the transaction amounts and the number of transactions for the given window size\n",
    "        SUM_AMOUNT_TX_WINDOW=customer_transactions['TX_AMOUNT'].rolling(str(window_size)+'d').sum()\n",
    "        NB_TX_WINDOW=customer_transactions['TX_AMOUNT'].rolling(str(window_size)+'d').count()\n",
    "    \n",
    "        # Compute the average transaction amount for the given window size\n",
    "        # NB_TX_WINDOW is always >0 since current transaction is always included\n",
    "        AVG_AMOUNT_TX_WINDOW=SUM_AMOUNT_TX_WINDOW/NB_TX_WINDOW\n",
    "    \n",
    "        # Save feature values\n",
    "        customer_transactions['CUSTOMER_ID_NB_TX_'+str(window_size)+'DAY_WINDOW']=list(NB_TX_WINDOW)\n",
    "        customer_transactions['CUSTOMER_ID_AVG_AMOUNT_'+str(window_size)+'DAY_WINDOW']=list(AVG_AMOUNT_TX_WINDOW)\n",
    "    \n",
    "    # Reindex according to transaction IDs\n",
    "    customer_transactions.index=customer_transactions.TRANSACTION_ID\n",
    "        \n",
    "    # And return the dataframe with the new features\n",
    "    return customer_transactions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us compute these aggregates for the first customer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>TX_DURING_NIGHT</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1758</th>\n",
       "      <td>1758</td>\n",
       "      <td>2018-04-01 07:19:05</td>\n",
       "      <td>0</td>\n",
       "      <td>6076</td>\n",
       "      <td>123.59</td>\n",
       "      <td>26345</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>123.590000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>123.590000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>123.590000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8275</th>\n",
       "      <td>8275</td>\n",
       "      <td>2018-04-01 18:00:16</td>\n",
       "      <td>0</td>\n",
       "      <td>858</td>\n",
       "      <td>77.34</td>\n",
       "      <td>64816</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>100.465000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>100.465000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>100.465000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8640</th>\n",
       "      <td>8640</td>\n",
       "      <td>2018-04-01 19:02:02</td>\n",
       "      <td>0</td>\n",
       "      <td>6698</td>\n",
       "      <td>46.51</td>\n",
       "      <td>68522</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>82.480000</td>\n",
       "      <td>3.0</td>\n",
       "      <td>82.480000</td>\n",
       "      <td>3.0</td>\n",
       "      <td>82.480000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12169</th>\n",
       "      <td>12169</td>\n",
       "      <td>2018-04-02 08:51:06</td>\n",
       "      <td>0</td>\n",
       "      <td>6569</td>\n",
       "      <td>54.72</td>\n",
       "      <td>118266</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>59.523333</td>\n",
       "      <td>4.0</td>\n",
       "      <td>75.540000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>75.540000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15764</th>\n",
       "      <td>15764</td>\n",
       "      <td>2018-04-02 14:05:38</td>\n",
       "      <td>0</td>\n",
       "      <td>7707</td>\n",
       "      <td>63.30</td>\n",
       "      <td>137138</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>60.467500</td>\n",
       "      <td>5.0</td>\n",
       "      <td>73.092000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>73.092000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1750390</th>\n",
       "      <td>1750390</td>\n",
       "      <td>2018-09-30 13:38:41</td>\n",
       "      <td>0</td>\n",
       "      <td>3096</td>\n",
       "      <td>38.23</td>\n",
       "      <td>15773921</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>64.388000</td>\n",
       "      <td>28.0</td>\n",
       "      <td>57.306429</td>\n",
       "      <td>89.0</td>\n",
       "      <td>63.097640</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1750758</th>\n",
       "      <td>1750758</td>\n",
       "      <td>2018-09-30 14:10:21</td>\n",
       "      <td>0</td>\n",
       "      <td>9441</td>\n",
       "      <td>43.60</td>\n",
       "      <td>15775821</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>60.923333</td>\n",
       "      <td>29.0</td>\n",
       "      <td>56.833793</td>\n",
       "      <td>89.0</td>\n",
       "      <td>62.433933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1751039</th>\n",
       "      <td>1751039</td>\n",
       "      <td>2018-09-30 14:34:30</td>\n",
       "      <td>0</td>\n",
       "      <td>1138</td>\n",
       "      <td>69.69</td>\n",
       "      <td>15777270</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>62.175714</td>\n",
       "      <td>29.0</td>\n",
       "      <td>57.872414</td>\n",
       "      <td>90.0</td>\n",
       "      <td>62.514556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1751272</th>\n",
       "      <td>1751272</td>\n",
       "      <td>2018-09-30 14:54:59</td>\n",
       "      <td>0</td>\n",
       "      <td>9441</td>\n",
       "      <td>91.26</td>\n",
       "      <td>15778499</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>65.811250</td>\n",
       "      <td>30.0</td>\n",
       "      <td>58.985333</td>\n",
       "      <td>90.0</td>\n",
       "      <td>61.882333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1751455</th>\n",
       "      <td>1751455</td>\n",
       "      <td>2018-09-30 15:11:37</td>\n",
       "      <td>0</td>\n",
       "      <td>2746</td>\n",
       "      <td>27.90</td>\n",
       "      <td>15779497</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>61.598889</td>\n",
       "      <td>31.0</td>\n",
       "      <td>57.982581</td>\n",
       "      <td>91.0</td>\n",
       "      <td>61.508901</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "TRANSACTION_ID                                                                 \n",
       "1758                      1758 2018-04-01 07:19:05            0         6076   \n",
       "8275                      8275 2018-04-01 18:00:16            0          858   \n",
       "8640                      8640 2018-04-01 19:02:02            0         6698   \n",
       "12169                    12169 2018-04-02 08:51:06            0         6569   \n",
       "15764                    15764 2018-04-02 14:05:38            0         7707   \n",
       "...                        ...                 ...          ...          ...   \n",
       "1750390                1750390 2018-09-30 13:38:41            0         3096   \n",
       "1750758                1750758 2018-09-30 14:10:21            0         9441   \n",
       "1751039                1751039 2018-09-30 14:34:30            0         1138   \n",
       "1751272                1751272 2018-09-30 14:54:59            0         9441   \n",
       "1751455                1751455 2018-09-30 15:11:37            0         2746   \n",
       "\n",
       "                TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "TRANSACTION_ID                                                       \n",
       "1758               123.59            26345             0         0   \n",
       "8275                77.34            64816             0         0   \n",
       "8640                46.51            68522             0         0   \n",
       "12169               54.72           118266             1         0   \n",
       "15764               63.30           137138             1         0   \n",
       "...                   ...              ...           ...       ...   \n",
       "1750390             38.23         15773921           182         0   \n",
       "1750758             43.60         15775821           182         0   \n",
       "1751039             69.69         15777270           182         0   \n",
       "1751272             91.26         15778499           182         0   \n",
       "1751455             27.90         15779497           182         0   \n",
       "\n",
       "                TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  TX_DURING_NIGHT  \\\n",
       "TRANSACTION_ID                                                          \n",
       "1758                            0                  1                0   \n",
       "8275                            0                  1                0   \n",
       "8640                            0                  1                0   \n",
       "12169                           0                  0                0   \n",
       "15764                           0                  0                0   \n",
       "...                           ...                ...              ...   \n",
       "1750390                         0                  1                0   \n",
       "1750758                         0                  1                0   \n",
       "1751039                         0                  1                0   \n",
       "1751272                         0                  1                0   \n",
       "1751455                         0                  1                0   \n",
       "\n",
       "                CUSTOMER_ID_NB_TX_1DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                  \n",
       "1758                                      1.0   \n",
       "8275                                      2.0   \n",
       "8640                                      3.0   \n",
       "12169                                     3.0   \n",
       "15764                                     4.0   \n",
       "...                                       ...   \n",
       "1750390                                   5.0   \n",
       "1750758                                   6.0   \n",
       "1751039                                   7.0   \n",
       "1751272                                   8.0   \n",
       "1751455                                   9.0   \n",
       "\n",
       "                CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                       \n",
       "1758                                    123.590000   \n",
       "8275                                    100.465000   \n",
       "8640                                     82.480000   \n",
       "12169                                    59.523333   \n",
       "15764                                    60.467500   \n",
       "...                                            ...   \n",
       "1750390                                  64.388000   \n",
       "1750758                                  60.923333   \n",
       "1751039                                  62.175714   \n",
       "1751272                                  65.811250   \n",
       "1751455                                  61.598889   \n",
       "\n",
       "                CUSTOMER_ID_NB_TX_7DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                  \n",
       "1758                                      1.0   \n",
       "8275                                      2.0   \n",
       "8640                                      3.0   \n",
       "12169                                     4.0   \n",
       "15764                                     5.0   \n",
       "...                                       ...   \n",
       "1750390                                  28.0   \n",
       "1750758                                  29.0   \n",
       "1751039                                  29.0   \n",
       "1751272                                  30.0   \n",
       "1751455                                  31.0   \n",
       "\n",
       "                CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                       \n",
       "1758                                    123.590000   \n",
       "8275                                    100.465000   \n",
       "8640                                     82.480000   \n",
       "12169                                    75.540000   \n",
       "15764                                    73.092000   \n",
       "...                                            ...   \n",
       "1750390                                  57.306429   \n",
       "1750758                                  56.833793   \n",
       "1751039                                  57.872414   \n",
       "1751272                                  58.985333   \n",
       "1751455                                  57.982581   \n",
       "\n",
       "                CUSTOMER_ID_NB_TX_30DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                   \n",
       "1758                                       1.0   \n",
       "8275                                       2.0   \n",
       "8640                                       3.0   \n",
       "12169                                      4.0   \n",
       "15764                                      5.0   \n",
       "...                                        ...   \n",
       "1750390                                   89.0   \n",
       "1750758                                   89.0   \n",
       "1751039                                   90.0   \n",
       "1751272                                   90.0   \n",
       "1751455                                   91.0   \n",
       "\n",
       "                CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW  \n",
       "TRANSACTION_ID                                       \n",
       "1758                                     123.590000  \n",
       "8275                                     100.465000  \n",
       "8640                                      82.480000  \n",
       "12169                                     75.540000  \n",
       "15764                                     73.092000  \n",
       "...                                             ...  \n",
       "1750390                                   63.097640  \n",
       "1750758                                   62.433933  \n",
       "1751039                                   62.514556  \n",
       "1751272                                   61.882333  \n",
       "1751455                                   61.508901  \n",
       "\n",
       "[384 rows x 17 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spending_behaviour_customer_0=get_customer_spending_behaviour_features(transactions_df[transactions_df.CUSTOMER_ID==0])\n",
    "spending_behaviour_customer_0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that the new features are consistent with the customer profile (see the previous notebook). For customer 0, the mean amount was `mean_amount`=62.26, and the transaction frequency was `mean_nb_tx_per_day`=2.18. These values are indeed closely matched by the features `CUSTOMER_ID_NB_TX_30DAY_WINDOW` and `CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW`, especially after 30 days.\n",
    "\n",
    "Let us now generate these features for all customers. This is straightforward using the Panda `groupby` and `apply` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1min 2s, sys: 1.21 s, total: 1min 3s\n",
      "Wall time: 1min 7s\n"
     ]
    }
   ],
   "source": [
    "%time transactions_df=transactions_df.groupby('CUSTOMER_ID').apply(lambda x: get_customer_spending_behaviour_features(x, windows_size_in_days=[1,7,30]))\n",
    "transactions_df=transactions_df.sort_values('TX_DATETIME').reset_index(drop=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>TX_DURING_NIGHT</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2018-04-01 00:00:31</td>\n",
       "      <td>596</td>\n",
       "      <td>3156</td>\n",
       "      <td>57.16</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.160000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.160000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.160000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2018-04-01 00:02:10</td>\n",
       "      <td>4961</td>\n",
       "      <td>3412</td>\n",
       "      <td>81.51</td>\n",
       "      <td>130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>81.510000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>81.510000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>81.510000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2018-04-01 00:07:56</td>\n",
       "      <td>2</td>\n",
       "      <td>1365</td>\n",
       "      <td>146.00</td>\n",
       "      <td>476</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>146.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>146.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>146.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2018-04-01 00:09:29</td>\n",
       "      <td>4128</td>\n",
       "      <td>8737</td>\n",
       "      <td>64.49</td>\n",
       "      <td>569</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>64.490000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>64.490000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>64.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>2018-04-01 00:10:34</td>\n",
       "      <td>927</td>\n",
       "      <td>9906</td>\n",
       "      <td>50.99</td>\n",
       "      <td>634</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>50.990000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>50.990000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>50.990000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754150</th>\n",
       "      <td>1754150</td>\n",
       "      <td>2018-09-30 23:56:36</td>\n",
       "      <td>161</td>\n",
       "      <td>655</td>\n",
       "      <td>54.24</td>\n",
       "      <td>15810996</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>75.280000</td>\n",
       "      <td>12.0</td>\n",
       "      <td>67.047500</td>\n",
       "      <td>72.0</td>\n",
       "      <td>69.521111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754151</th>\n",
       "      <td>1754151</td>\n",
       "      <td>2018-09-30 23:57:38</td>\n",
       "      <td>4342</td>\n",
       "      <td>6181</td>\n",
       "      <td>1.23</td>\n",
       "      <td>15811058</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.230000</td>\n",
       "      <td>21.0</td>\n",
       "      <td>22.173810</td>\n",
       "      <td>93.0</td>\n",
       "      <td>24.780753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754152</th>\n",
       "      <td>1754152</td>\n",
       "      <td>2018-09-30 23:58:21</td>\n",
       "      <td>618</td>\n",
       "      <td>1502</td>\n",
       "      <td>6.62</td>\n",
       "      <td>15811101</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>7.368000</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7.400476</td>\n",
       "      <td>65.0</td>\n",
       "      <td>7.864462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754153</th>\n",
       "      <td>1754153</td>\n",
       "      <td>2018-09-30 23:59:52</td>\n",
       "      <td>4056</td>\n",
       "      <td>3067</td>\n",
       "      <td>55.40</td>\n",
       "      <td>15811192</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>100.696667</td>\n",
       "      <td>16.0</td>\n",
       "      <td>107.052500</td>\n",
       "      <td>51.0</td>\n",
       "      <td>102.919608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754154</th>\n",
       "      <td>1754154</td>\n",
       "      <td>2018-09-30 23:59:57</td>\n",
       "      <td>3542</td>\n",
       "      <td>9849</td>\n",
       "      <td>23.59</td>\n",
       "      <td>15811197</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>41.304000</td>\n",
       "      <td>24.0</td>\n",
       "      <td>35.308333</td>\n",
       "      <td>119.0</td>\n",
       "      <td>37.251513</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1754155 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "0                     0 2018-04-01 00:00:31          596         3156   \n",
       "1                     1 2018-04-01 00:02:10         4961         3412   \n",
       "2                     2 2018-04-01 00:07:56            2         1365   \n",
       "3                     3 2018-04-01 00:09:29         4128         8737   \n",
       "4                     4 2018-04-01 00:10:34          927         9906   \n",
       "...                 ...                 ...          ...          ...   \n",
       "1754150         1754150 2018-09-30 23:56:36          161          655   \n",
       "1754151         1754151 2018-09-30 23:57:38         4342         6181   \n",
       "1754152         1754152 2018-09-30 23:58:21          618         1502   \n",
       "1754153         1754153 2018-09-30 23:59:52         4056         3067   \n",
       "1754154         1754154 2018-09-30 23:59:57         3542         9849   \n",
       "\n",
       "         TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "0            57.16               31             0         0   \n",
       "1            81.51              130             0         0   \n",
       "2           146.00              476             0         0   \n",
       "3            64.49              569             0         0   \n",
       "4            50.99              634             0         0   \n",
       "...            ...              ...           ...       ...   \n",
       "1754150      54.24         15810996           182         0   \n",
       "1754151       1.23         15811058           182         0   \n",
       "1754152       6.62         15811101           182         0   \n",
       "1754153      55.40         15811192           182         0   \n",
       "1754154      23.59         15811197           182         0   \n",
       "\n",
       "         TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  TX_DURING_NIGHT  \\\n",
       "0                        0                  1                1   \n",
       "1                        0                  1                1   \n",
       "2                        0                  1                1   \n",
       "3                        0                  1                1   \n",
       "4                        0                  1                1   \n",
       "...                    ...                ...              ...   \n",
       "1754150                  0                  1                0   \n",
       "1754151                  0                  1                0   \n",
       "1754152                  0                  1                0   \n",
       "1754153                  0                  1                0   \n",
       "1754154                  0                  1                0   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_1DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW  \\\n",
       "0                                  1.0                           57.160000   \n",
       "1                                  1.0                           81.510000   \n",
       "2                                  1.0                          146.000000   \n",
       "3                                  1.0                           64.490000   \n",
       "4                                  1.0                           50.990000   \n",
       "...                                ...                                 ...   \n",
       "1754150                            2.0                           75.280000   \n",
       "1754151                            1.0                            1.230000   \n",
       "1754152                            5.0                            7.368000   \n",
       "1754153                            3.0                          100.696667   \n",
       "1754154                            5.0                           41.304000   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_7DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW  \\\n",
       "0                                  1.0                           57.160000   \n",
       "1                                  1.0                           81.510000   \n",
       "2                                  1.0                          146.000000   \n",
       "3                                  1.0                           64.490000   \n",
       "4                                  1.0                           50.990000   \n",
       "...                                ...                                 ...   \n",
       "1754150                           12.0                           67.047500   \n",
       "1754151                           21.0                           22.173810   \n",
       "1754152                           21.0                            7.400476   \n",
       "1754153                           16.0                          107.052500   \n",
       "1754154                           24.0                           35.308333   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_30DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW  \n",
       "0                                   1.0                            57.160000  \n",
       "1                                   1.0                            81.510000  \n",
       "2                                   1.0                           146.000000  \n",
       "3                                   1.0                            64.490000  \n",
       "4                                   1.0                            50.990000  \n",
       "...                                 ...                                  ...  \n",
       "1754150                            72.0                            69.521111  \n",
       "1754151                            93.0                            24.780753  \n",
       "1754152                            65.0                             7.864462  \n",
       "1754153                            51.0                           102.919608  \n",
       "1754154                           119.0                            37.251513  \n",
       "\n",
       "[1754155 rows x 17 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Terminal ID transformations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let us proceed with the terminal ID transformations. The main goal will be to extract a *risk score*, that assesses the exposure of a given terminal ID to fraudulent transactions. The risk score will be defined as the average number of fraudulent transactions that occurred on a terminal ID over a time window. As for customer ID transformations, we will use three window sizes, of 1, 7, and 30 days.\n",
    "\n",
    "Contrary to customer ID transformations, the time windows will not directly precede a given transaction. Instead, they will be shifted back by a *delay period*. The delay period accounts for the fact that, in practice, the fraudulent transactions are only discovered after a fraud investigation or a customer complaint. Hence, the fraudulent labels, which are needed to compute the risk score, are only available after this delay period. To a first approximation, this delay period will be set to one week. The motivations for the delay period will be further argued in [Chapter 5, Validation strategies](Validation_Strategies). \n",
    "\n",
    "Let us perform the computation of the risk scores by defining a `get_count_risk_rolling_window` function. The function takes as inputs the DataFrame of transactions for a given terminal ID, the delay period, and a list of window sizes. In the first stage, the number of transactions and fraudulent transactions are computed for the delay period (`NB_TX_DELAY` and `NB_FRAUD_DELAY`). In the second stage, the number of transactions and fraudulent transactions are computed for each window size plus the delay period (`NB_TX_DELAY_WINDOW` and `NB_FRAUD_DELAY_WINDOW`). The number of transactions and fraudulent transactions that occurred for a given window size, shifted back by the delay period, is then obtained by simply computing the differences of the quantities obtained for the delay period, and the window size plus delay period:\n",
    "\n",
    "```\n",
    "NB_FRAUD_WINDOW=NB_FRAUD_DELAY_WINDOW-NB_FRAUD_DELAY\n",
    "NB_TX_WINDOW=NB_TX_DELAY_WINDOW-NB_TX_DELAY\n",
    "```\n",
    "\n",
    "The risk score is finally obtained by computing the proportion of fraudulent transactions for each window size (or 0 if no transaction occurred for the given window):\n",
    "\n",
    "```\n",
    "RISK_WINDOW=NB_FRAUD_WINDOW/NB_TX_WINDOW\n",
    "```\n",
    "\n",
    "Additionally to the risk score, the function also returns the number of transactions for each window size. This results in the addition of six new features: The risk and number of transactions, for three window sizes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_count_risk_rolling_window(terminal_transactions, delay_period=7, windows_size_in_days=[1,7,30], feature=\"TERMINAL_ID\"):\n",
    "    \n",
    "    terminal_transactions=terminal_transactions.sort_values('TX_DATETIME')\n",
    "    \n",
    "    terminal_transactions.index=terminal_transactions.TX_DATETIME\n",
    "    \n",
    "    NB_FRAUD_DELAY=terminal_transactions['TX_FRAUD'].rolling(str(delay_period)+'d').sum()\n",
    "    NB_TX_DELAY=terminal_transactions['TX_FRAUD'].rolling(str(delay_period)+'d').count()\n",
    "    \n",
    "    for window_size in windows_size_in_days:\n",
    "    \n",
    "        NB_FRAUD_DELAY_WINDOW=terminal_transactions['TX_FRAUD'].rolling(str(delay_period+window_size)+'d').sum()\n",
    "        NB_TX_DELAY_WINDOW=terminal_transactions['TX_FRAUD'].rolling(str(delay_period+window_size)+'d').count()\n",
    "    \n",
    "        NB_FRAUD_WINDOW=NB_FRAUD_DELAY_WINDOW-NB_FRAUD_DELAY\n",
    "        NB_TX_WINDOW=NB_TX_DELAY_WINDOW-NB_TX_DELAY\n",
    "    \n",
    "        RISK_WINDOW=NB_FRAUD_WINDOW/NB_TX_WINDOW\n",
    "        \n",
    "        terminal_transactions[feature+'_NB_TX_'+str(window_size)+'DAY_WINDOW']=list(NB_TX_WINDOW)\n",
    "        terminal_transactions[feature+'_RISK_'+str(window_size)+'DAY_WINDOW']=list(RISK_WINDOW)\n",
    "        \n",
    "    terminal_transactions.index=terminal_transactions.TRANSACTION_ID\n",
    "    \n",
    "    # Replace NA values with 0 (all undefined risk scores where NB_TX_WINDOW is 0) \n",
    "    terminal_transactions.fillna(0,inplace=True)\n",
    "    \n",
    "    return terminal_transactions\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>TX_DURING_NIGHT</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3527</th>\n",
       "      <td>3527</td>\n",
       "      <td>2018-04-01 10:17:43</td>\n",
       "      <td>3774</td>\n",
       "      <td>3059</td>\n",
       "      <td>225.41</td>\n",
       "      <td>37063</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>158.073333</td>\n",
       "      <td>3.0</td>\n",
       "      <td>158.073333</td>\n",
       "      <td>3.0</td>\n",
       "      <td>158.073333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5789</th>\n",
       "      <td>5790</td>\n",
       "      <td>2018-04-01 13:31:48</td>\n",
       "      <td>4944</td>\n",
       "      <td>6050</td>\n",
       "      <td>222.26</td>\n",
       "      <td>48708</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>127.605000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>127.605000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>127.605000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6549</th>\n",
       "      <td>6549</td>\n",
       "      <td>2018-04-01 14:42:02</td>\n",
       "      <td>4625</td>\n",
       "      <td>9102</td>\n",
       "      <td>226.40</td>\n",
       "      <td>52922</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>167.165000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>167.165000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>167.165000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9583</th>\n",
       "      <td>9583</td>\n",
       "      <td>2018-04-02 01:01:05</td>\n",
       "      <td>3814</td>\n",
       "      <td>6893</td>\n",
       "      <td>59.15</td>\n",
       "      <td>90065</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>29.138333</td>\n",
       "      <td>6.0</td>\n",
       "      <td>29.138333</td>\n",
       "      <td>6.0</td>\n",
       "      <td>29.138333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10356</th>\n",
       "      <td>10355</td>\n",
       "      <td>2018-04-02 05:03:35</td>\n",
       "      <td>2513</td>\n",
       "      <td>1143</td>\n",
       "      <td>222.04</td>\n",
       "      <td>104615</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5.0</td>\n",
       "      <td>123.740000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>123.740000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>123.740000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1753524</th>\n",
       "      <td>1753524</td>\n",
       "      <td>2018-09-30 19:51:48</td>\n",
       "      <td>1671</td>\n",
       "      <td>3192</td>\n",
       "      <td>128.60</td>\n",
       "      <td>15796308</td>\n",
       "      <td>182</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>138.358333</td>\n",
       "      <td>25.0</td>\n",
       "      <td>106.957200</td>\n",
       "      <td>82.0</td>\n",
       "      <td>75.621341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1753600</th>\n",
       "      <td>1753600</td>\n",
       "      <td>2018-09-30 20:09:00</td>\n",
       "      <td>4166</td>\n",
       "      <td>632</td>\n",
       "      <td>17.39</td>\n",
       "      <td>15797340</td>\n",
       "      <td>182</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>19.766667</td>\n",
       "      <td>19.0</td>\n",
       "      <td>15.984737</td>\n",
       "      <td>86.0</td>\n",
       "      <td>15.846512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1753673</th>\n",
       "      <td>1753673</td>\n",
       "      <td>2018-09-30 20:30:52</td>\n",
       "      <td>4097</td>\n",
       "      <td>1558</td>\n",
       "      <td>24.04</td>\n",
       "      <td>15798652</td>\n",
       "      <td>182</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>23.050000</td>\n",
       "      <td>16.0</td>\n",
       "      <td>40.440625</td>\n",
       "      <td>63.0</td>\n",
       "      <td>41.877460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754014</th>\n",
       "      <td>1754014</td>\n",
       "      <td>2018-09-30 22:27:04</td>\n",
       "      <td>100</td>\n",
       "      <td>8604</td>\n",
       "      <td>73.85</td>\n",
       "      <td>15805624</td>\n",
       "      <td>182</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>48.010000</td>\n",
       "      <td>26.0</td>\n",
       "      <td>30.384231</td>\n",
       "      <td>103.0</td>\n",
       "      <td>23.627184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754017</th>\n",
       "      <td>1754018</td>\n",
       "      <td>2018-09-30 22:28:01</td>\n",
       "      <td>4677</td>\n",
       "      <td>8935</td>\n",
       "      <td>45.85</td>\n",
       "      <td>15805681</td>\n",
       "      <td>182</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>39.078000</td>\n",
       "      <td>19.0</td>\n",
       "      <td>35.133684</td>\n",
       "      <td>85.0</td>\n",
       "      <td>37.656000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>14681 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "3527               3527 2018-04-01 10:17:43         3774         3059   \n",
       "5789               5790 2018-04-01 13:31:48         4944         6050   \n",
       "6549               6549 2018-04-01 14:42:02         4625         9102   \n",
       "9583               9583 2018-04-02 01:01:05         3814         6893   \n",
       "10356             10355 2018-04-02 05:03:35         2513         1143   \n",
       "...                 ...                 ...          ...          ...   \n",
       "1753524         1753524 2018-09-30 19:51:48         1671         3192   \n",
       "1753600         1753600 2018-09-30 20:09:00         4166          632   \n",
       "1753673         1753673 2018-09-30 20:30:52         4097         1558   \n",
       "1754014         1754014 2018-09-30 22:27:04          100         8604   \n",
       "1754017         1754018 2018-09-30 22:28:01         4677         8935   \n",
       "\n",
       "         TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "3527        225.41            37063             0         1   \n",
       "5789        222.26            48708             0         1   \n",
       "6549        226.40            52922             0         1   \n",
       "9583         59.15            90065             1         1   \n",
       "10356       222.04           104615             1         1   \n",
       "...            ...              ...           ...       ...   \n",
       "1753524     128.60         15796308           182         1   \n",
       "1753600      17.39         15797340           182         1   \n",
       "1753673      24.04         15798652           182         1   \n",
       "1754014      73.85         15805624           182         1   \n",
       "1754017      45.85         15805681           182         1   \n",
       "\n",
       "         TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  TX_DURING_NIGHT  \\\n",
       "3527                     1                  1                0   \n",
       "5789                     1                  1                0   \n",
       "6549                     1                  1                0   \n",
       "9583                     3                  0                1   \n",
       "10356                    1                  0                1   \n",
       "...                    ...                ...              ...   \n",
       "1753524                  3                  1                0   \n",
       "1753600                  2                  1                0   \n",
       "1753673                  2                  1                0   \n",
       "1754014                  3                  1                0   \n",
       "1754017                  2                  1                0   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_1DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_1DAY_WINDOW  \\\n",
       "3527                               3.0                          158.073333   \n",
       "5789                               2.0                          127.605000   \n",
       "6549                               4.0                          167.165000   \n",
       "9583                               6.0                           29.138333   \n",
       "10356                              5.0                          123.740000   \n",
       "...                                ...                                 ...   \n",
       "1753524                            6.0                          138.358333   \n",
       "1753600                            3.0                           19.766667   \n",
       "1753673                            3.0                           23.050000   \n",
       "1754014                            2.0                           48.010000   \n",
       "1754017                            5.0                           39.078000   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_7DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW  \\\n",
       "3527                               3.0                          158.073333   \n",
       "5789                               2.0                          127.605000   \n",
       "6549                               4.0                          167.165000   \n",
       "9583                               6.0                           29.138333   \n",
       "10356                              5.0                          123.740000   \n",
       "...                                ...                                 ...   \n",
       "1753524                           25.0                          106.957200   \n",
       "1753600                           19.0                           15.984737   \n",
       "1753673                           16.0                           40.440625   \n",
       "1754014                           26.0                           30.384231   \n",
       "1754017                           19.0                           35.133684   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_30DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW  \n",
       "3527                                3.0                           158.073333  \n",
       "5789                                2.0                           127.605000  \n",
       "6549                                4.0                           167.165000  \n",
       "9583                                6.0                            29.138333  \n",
       "10356                               5.0                           123.740000  \n",
       "...                                 ...                                  ...  \n",
       "1753524                            82.0                            75.621341  \n",
       "1753600                            86.0                            15.846512  \n",
       "1753673                            63.0                            41.877460  \n",
       "1754014                           103.0                            23.627184  \n",
       "1754017                            85.0                            37.656000  \n",
       "\n",
       "[14681 rows x 17 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_df[transactions_df.TX_FRAUD==1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us compute these six features for the first terminal ID containing at least one fraud:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3156"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the first terminal ID that contains frauds\n",
    "transactions_df[transactions_df.TX_FRAUD==0].TERMINAL_ID[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>...</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_1DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_1DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_7DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_30DAY_WINDOW</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3527</th>\n",
       "      <td>3527</td>\n",
       "      <td>2018-04-01 10:17:43</td>\n",
       "      <td>3774</td>\n",
       "      <td>3059</td>\n",
       "      <td>225.41</td>\n",
       "      <td>37063</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>3.0</td>\n",
       "      <td>158.073333</td>\n",
       "      <td>3.0</td>\n",
       "      <td>158.073333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4732</th>\n",
       "      <td>4732</td>\n",
       "      <td>2018-04-01 11:59:14</td>\n",
       "      <td>55</td>\n",
       "      <td>3059</td>\n",
       "      <td>36.28</td>\n",
       "      <td>43154</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>2.0</td>\n",
       "      <td>35.670000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>35.670000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16216</th>\n",
       "      <td>16216</td>\n",
       "      <td>2018-04-02 14:47:34</td>\n",
       "      <td>4879</td>\n",
       "      <td>3059</td>\n",
       "      <td>105.00</td>\n",
       "      <td>139654</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>10.0</td>\n",
       "      <td>76.010000</td>\n",
       "      <td>10.0</td>\n",
       "      <td>76.010000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18249</th>\n",
       "      <td>18249</td>\n",
       "      <td>2018-04-02 19:08:10</td>\n",
       "      <td>2263</td>\n",
       "      <td>3059</td>\n",
       "      <td>90.89</td>\n",
       "      <td>155290</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>50.458571</td>\n",
       "      <td>7.0</td>\n",
       "      <td>50.458571</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26512</th>\n",
       "      <td>26512</td>\n",
       "      <td>2018-04-03 15:44:49</td>\n",
       "      <td>4879</td>\n",
       "      <td>3059</td>\n",
       "      <td>58.51</td>\n",
       "      <td>229489</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>14.0</td>\n",
       "      <td>71.070000</td>\n",
       "      <td>14.0</td>\n",
       "      <td>71.070000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1697944</th>\n",
       "      <td>1697944</td>\n",
       "      <td>2018-09-25 05:32:56</td>\n",
       "      <td>402</td>\n",
       "      <td>3059</td>\n",
       "      <td>57.30</td>\n",
       "      <td>15312776</td>\n",
       "      <td>177</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>14.0</td>\n",
       "      <td>65.167857</td>\n",
       "      <td>46.0</td>\n",
       "      <td>68.163261</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1701971</th>\n",
       "      <td>1701971</td>\n",
       "      <td>2018-09-25 12:30:54</td>\n",
       "      <td>1035</td>\n",
       "      <td>3059</td>\n",
       "      <td>7.56</td>\n",
       "      <td>15337854</td>\n",
       "      <td>177</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>23.0</td>\n",
       "      <td>7.052174</td>\n",
       "      <td>107.0</td>\n",
       "      <td>6.763738</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1704512</th>\n",
       "      <td>1704512</td>\n",
       "      <td>2018-09-25 16:37:41</td>\n",
       "      <td>1519</td>\n",
       "      <td>3059</td>\n",
       "      <td>35.79</td>\n",
       "      <td>15352661</td>\n",
       "      <td>177</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>41.404286</td>\n",
       "      <td>30.0</td>\n",
       "      <td>46.780000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1731937</th>\n",
       "      <td>1731937</td>\n",
       "      <td>2018-09-28 14:30:31</td>\n",
       "      <td>1534</td>\n",
       "      <td>3059</td>\n",
       "      <td>81.39</td>\n",
       "      <td>15604231</td>\n",
       "      <td>180</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>18.0</td>\n",
       "      <td>69.477778</td>\n",
       "      <td>89.0</td>\n",
       "      <td>63.906629</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1740901</th>\n",
       "      <td>1740901</td>\n",
       "      <td>2018-09-29 13:35:17</td>\n",
       "      <td>118</td>\n",
       "      <td>3059</td>\n",
       "      <td>90.96</td>\n",
       "      <td>15687317</td>\n",
       "      <td>181</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>35.0</td>\n",
       "      <td>104.233714</td>\n",
       "      <td>98.0</td>\n",
       "      <td>91.407143</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>193 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "TRANSACTION_ID                                                                 \n",
       "3527                      3527 2018-04-01 10:17:43         3774         3059   \n",
       "4732                      4732 2018-04-01 11:59:14           55         3059   \n",
       "16216                    16216 2018-04-02 14:47:34         4879         3059   \n",
       "18249                    18249 2018-04-02 19:08:10         2263         3059   \n",
       "26512                    26512 2018-04-03 15:44:49         4879         3059   \n",
       "...                        ...                 ...          ...          ...   \n",
       "1697944                1697944 2018-09-25 05:32:56          402         3059   \n",
       "1701971                1701971 2018-09-25 12:30:54         1035         3059   \n",
       "1704512                1704512 2018-09-25 16:37:41         1519         3059   \n",
       "1731937                1731937 2018-09-28 14:30:31         1534         3059   \n",
       "1740901                1740901 2018-09-29 13:35:17          118         3059   \n",
       "\n",
       "                TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "TRANSACTION_ID                                                       \n",
       "3527               225.41            37063             0         1   \n",
       "4732                36.28            43154             0         0   \n",
       "16216              105.00           139654             1         0   \n",
       "18249               90.89           155290             1         0   \n",
       "26512               58.51           229489             2         0   \n",
       "...                   ...              ...           ...       ...   \n",
       "1697944             57.30         15312776           177         0   \n",
       "1701971              7.56         15337854           177         0   \n",
       "1704512             35.79         15352661           177         0   \n",
       "1731937             81.39         15604231           180         0   \n",
       "1740901             90.96         15687317           181         0   \n",
       "\n",
       "                TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  ...  \\\n",
       "TRANSACTION_ID                                        ...   \n",
       "3527                            1                  1  ...   \n",
       "4732                            0                  1  ...   \n",
       "16216                           0                  0  ...   \n",
       "18249                           0                  0  ...   \n",
       "26512                           0                  0  ...   \n",
       "...                           ...                ...  ...   \n",
       "1697944                         0                  0  ...   \n",
       "1701971                         0                  0  ...   \n",
       "1704512                         0                  0  ...   \n",
       "1731937                         0                  0  ...   \n",
       "1740901                         0                  1  ...   \n",
       "\n",
       "                CUSTOMER_ID_NB_TX_7DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                  \n",
       "3527                                      3.0   \n",
       "4732                                      2.0   \n",
       "16216                                    10.0   \n",
       "18249                                     7.0   \n",
       "26512                                    14.0   \n",
       "...                                       ...   \n",
       "1697944                                  14.0   \n",
       "1701971                                  23.0   \n",
       "1704512                                   7.0   \n",
       "1731937                                  18.0   \n",
       "1740901                                  35.0   \n",
       "\n",
       "                CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                       \n",
       "3527                                    158.073333   \n",
       "4732                                     35.670000   \n",
       "16216                                    76.010000   \n",
       "18249                                    50.458571   \n",
       "26512                                    71.070000   \n",
       "...                                            ...   \n",
       "1697944                                  65.167857   \n",
       "1701971                                   7.052174   \n",
       "1704512                                  41.404286   \n",
       "1731937                                  69.477778   \n",
       "1740901                                 104.233714   \n",
       "\n",
       "                CUSTOMER_ID_NB_TX_30DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                   \n",
       "3527                                       3.0   \n",
       "4732                                       2.0   \n",
       "16216                                     10.0   \n",
       "18249                                      7.0   \n",
       "26512                                     14.0   \n",
       "...                                        ...   \n",
       "1697944                                   46.0   \n",
       "1701971                                  107.0   \n",
       "1704512                                   30.0   \n",
       "1731937                                   89.0   \n",
       "1740901                                   98.0   \n",
       "\n",
       "                CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                        \n",
       "3527                                     158.073333   \n",
       "4732                                      35.670000   \n",
       "16216                                     76.010000   \n",
       "18249                                     50.458571   \n",
       "26512                                     71.070000   \n",
       "...                                             ...   \n",
       "1697944                                   68.163261   \n",
       "1701971                                    6.763738   \n",
       "1704512                                   46.780000   \n",
       "1731937                                   63.906629   \n",
       "1740901                                   91.407143   \n",
       "\n",
       "                TERMINAL_ID_NB_TX_1DAY_WINDOW  TERMINAL_ID_RISK_1DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                                                \n",
       "3527                                      0.0                           0.0   \n",
       "4732                                      0.0                           0.0   \n",
       "16216                                     0.0                           0.0   \n",
       "18249                                     0.0                           0.0   \n",
       "26512                                     0.0                           0.0   \n",
       "...                                       ...                           ...   \n",
       "1697944                                   1.0                           0.0   \n",
       "1701971                                   2.0                           0.0   \n",
       "1704512                                   1.0                           0.0   \n",
       "1731937                                   1.0                           0.0   \n",
       "1740901                                   0.0                           0.0   \n",
       "\n",
       "                TERMINAL_ID_NB_TX_7DAY_WINDOW  TERMINAL_ID_RISK_7DAY_WINDOW  \\\n",
       "TRANSACTION_ID                                                                \n",
       "3527                                      0.0                           0.0   \n",
       "4732                                      0.0                           0.0   \n",
       "16216                                     0.0                           0.0   \n",
       "18249                                     0.0                           0.0   \n",
       "26512                                     0.0                           0.0   \n",
       "...                                       ...                           ...   \n",
       "1697944                                   9.0                           0.0   \n",
       "1701971                                  10.0                           0.0   \n",
       "1704512                                   9.0                           0.0   \n",
       "1731937                                   8.0                           0.0   \n",
       "1740901                                   7.0                           0.0   \n",
       "\n",
       "                TERMINAL_ID_NB_TX_30DAY_WINDOW  TERMINAL_ID_RISK_30DAY_WINDOW  \n",
       "TRANSACTION_ID                                                                 \n",
       "3527                                       0.0                            0.0  \n",
       "4732                                       0.0                            0.0  \n",
       "16216                                      0.0                            0.0  \n",
       "18249                                      0.0                            0.0  \n",
       "26512                                      0.0                            0.0  \n",
       "...                                        ...                            ...  \n",
       "1697944                                   36.0                            0.0  \n",
       "1701971                                   36.0                            0.0  \n",
       "1704512                                   36.0                            0.0  \n",
       "1731937                                   36.0                            0.0  \n",
       "1740901                                   36.0                            0.0  \n",
       "\n",
       "[193 rows x 23 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_count_risk_rolling_window(transactions_df[transactions_df.TERMINAL_ID==3059], delay_period=7, windows_size_in_days=[1,7,30])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check that the first fraud occurred on the 2018/09/10, and that risk scores only start being counted with a one-week delay. \n",
    "\n",
    "Let us finally generate these features for all terminals. This is straightforward using the Panda `groupby` and `apply` methods. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 27s, sys: 2.23 s, total: 2min 29s\n",
      "Wall time: 2min 41s\n"
     ]
    }
   ],
   "source": [
    "%time transactions_df=transactions_df.groupby('TERMINAL_ID').apply(lambda x: get_count_risk_rolling_window(x, delay_period=7, windows_size_in_days=[1,7,30], feature=\"TERMINAL_ID\"))\n",
    "transactions_df=transactions_df.sort_values('TX_DATETIME').reset_index(drop=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TRANSACTION_ID</th>\n",
       "      <th>TX_DATETIME</th>\n",
       "      <th>CUSTOMER_ID</th>\n",
       "      <th>TERMINAL_ID</th>\n",
       "      <th>TX_AMOUNT</th>\n",
       "      <th>TX_TIME_SECONDS</th>\n",
       "      <th>TX_TIME_DAYS</th>\n",
       "      <th>TX_FRAUD</th>\n",
       "      <th>TX_FRAUD_SCENARIO</th>\n",
       "      <th>TX_DURING_WEEKEND</th>\n",
       "      <th>...</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_1DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_1DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_7DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_7DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_NB_TX_30DAY_WINDOW</th>\n",
       "      <th>TERMINAL_ID_RISK_30DAY_WINDOW</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2018-04-01 00:00:31</td>\n",
       "      <td>596</td>\n",
       "      <td>3156</td>\n",
       "      <td>57.16</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.160000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>57.160000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2018-04-01 00:02:10</td>\n",
       "      <td>4961</td>\n",
       "      <td>3412</td>\n",
       "      <td>81.51</td>\n",
       "      <td>130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>81.510000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>81.510000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2018-04-01 00:07:56</td>\n",
       "      <td>2</td>\n",
       "      <td>1365</td>\n",
       "      <td>146.00</td>\n",
       "      <td>476</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>146.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>146.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>2018-04-01 00:09:29</td>\n",
       "      <td>4128</td>\n",
       "      <td>8737</td>\n",
       "      <td>64.49</td>\n",
       "      <td>569</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>64.490000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>64.490000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>2018-04-01 00:10:34</td>\n",
       "      <td>927</td>\n",
       "      <td>9906</td>\n",
       "      <td>50.99</td>\n",
       "      <td>634</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>50.990000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>50.990000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754150</th>\n",
       "      <td>1754150</td>\n",
       "      <td>2018-09-30 23:56:36</td>\n",
       "      <td>161</td>\n",
       "      <td>655</td>\n",
       "      <td>54.24</td>\n",
       "      <td>15810996</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>12.0</td>\n",
       "      <td>67.047500</td>\n",
       "      <td>72.0</td>\n",
       "      <td>69.521111</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754151</th>\n",
       "      <td>1754151</td>\n",
       "      <td>2018-09-30 23:57:38</td>\n",
       "      <td>4342</td>\n",
       "      <td>6181</td>\n",
       "      <td>1.23</td>\n",
       "      <td>15811058</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>21.0</td>\n",
       "      <td>22.173810</td>\n",
       "      <td>93.0</td>\n",
       "      <td>24.780753</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754152</th>\n",
       "      <td>1754152</td>\n",
       "      <td>2018-09-30 23:58:21</td>\n",
       "      <td>618</td>\n",
       "      <td>1502</td>\n",
       "      <td>6.62</td>\n",
       "      <td>15811101</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>21.0</td>\n",
       "      <td>7.400476</td>\n",
       "      <td>65.0</td>\n",
       "      <td>7.864462</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754153</th>\n",
       "      <td>1754153</td>\n",
       "      <td>2018-09-30 23:59:52</td>\n",
       "      <td>4056</td>\n",
       "      <td>3067</td>\n",
       "      <td>55.40</td>\n",
       "      <td>15811192</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>16.0</td>\n",
       "      <td>107.052500</td>\n",
       "      <td>51.0</td>\n",
       "      <td>102.919608</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1754154</th>\n",
       "      <td>1754154</td>\n",
       "      <td>2018-09-30 23:59:57</td>\n",
       "      <td>3542</td>\n",
       "      <td>9849</td>\n",
       "      <td>23.59</td>\n",
       "      <td>15811197</td>\n",
       "      <td>182</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>...</td>\n",
       "      <td>24.0</td>\n",
       "      <td>35.308333</td>\n",
       "      <td>119.0</td>\n",
       "      <td>37.251513</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>0.02439</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1754155 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         TRANSACTION_ID         TX_DATETIME  CUSTOMER_ID  TERMINAL_ID  \\\n",
       "0                     0 2018-04-01 00:00:31          596         3156   \n",
       "1                     1 2018-04-01 00:02:10         4961         3412   \n",
       "2                     2 2018-04-01 00:07:56            2         1365   \n",
       "3                     3 2018-04-01 00:09:29         4128         8737   \n",
       "4                     4 2018-04-01 00:10:34          927         9906   \n",
       "...                 ...                 ...          ...          ...   \n",
       "1754150         1754150 2018-09-30 23:56:36          161          655   \n",
       "1754151         1754151 2018-09-30 23:57:38         4342         6181   \n",
       "1754152         1754152 2018-09-30 23:58:21          618         1502   \n",
       "1754153         1754153 2018-09-30 23:59:52         4056         3067   \n",
       "1754154         1754154 2018-09-30 23:59:57         3542         9849   \n",
       "\n",
       "         TX_AMOUNT  TX_TIME_SECONDS  TX_TIME_DAYS  TX_FRAUD  \\\n",
       "0            57.16               31             0         0   \n",
       "1            81.51              130             0         0   \n",
       "2           146.00              476             0         0   \n",
       "3            64.49              569             0         0   \n",
       "4            50.99              634             0         0   \n",
       "...            ...              ...           ...       ...   \n",
       "1754150      54.24         15810996           182         0   \n",
       "1754151       1.23         15811058           182         0   \n",
       "1754152       6.62         15811101           182         0   \n",
       "1754153      55.40         15811192           182         0   \n",
       "1754154      23.59         15811197           182         0   \n",
       "\n",
       "         TX_FRAUD_SCENARIO  TX_DURING_WEEKEND  ...  \\\n",
       "0                        0                  1  ...   \n",
       "1                        0                  1  ...   \n",
       "2                        0                  1  ...   \n",
       "3                        0                  1  ...   \n",
       "4                        0                  1  ...   \n",
       "...                    ...                ...  ...   \n",
       "1754150                  0                  1  ...   \n",
       "1754151                  0                  1  ...   \n",
       "1754152                  0                  1  ...   \n",
       "1754153                  0                  1  ...   \n",
       "1754154                  0                  1  ...   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_7DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_7DAY_WINDOW  \\\n",
       "0                                  1.0                           57.160000   \n",
       "1                                  1.0                           81.510000   \n",
       "2                                  1.0                          146.000000   \n",
       "3                                  1.0                           64.490000   \n",
       "4                                  1.0                           50.990000   \n",
       "...                                ...                                 ...   \n",
       "1754150                           12.0                           67.047500   \n",
       "1754151                           21.0                           22.173810   \n",
       "1754152                           21.0                            7.400476   \n",
       "1754153                           16.0                          107.052500   \n",
       "1754154                           24.0                           35.308333   \n",
       "\n",
       "         CUSTOMER_ID_NB_TX_30DAY_WINDOW  CUSTOMER_ID_AVG_AMOUNT_30DAY_WINDOW  \\\n",
       "0                                   1.0                            57.160000   \n",
       "1                                   1.0                            81.510000   \n",
       "2                                   1.0                           146.000000   \n",
       "3                                   1.0                            64.490000   \n",
       "4                                   1.0                            50.990000   \n",
       "...                                 ...                                  ...   \n",
       "1754150                            72.0                            69.521111   \n",
       "1754151                            93.0                            24.780753   \n",
       "1754152                            65.0                             7.864462   \n",
       "1754153                            51.0                           102.919608   \n",
       "1754154                           119.0                            37.251513   \n",
       "\n",
       "         TERMINAL_ID_NB_TX_1DAY_WINDOW  TERMINAL_ID_RISK_1DAY_WINDOW  \\\n",
       "0                                  0.0                           0.0   \n",
       "1                                  0.0                           0.0   \n",
       "2                                  0.0                           0.0   \n",
       "3                                  0.0                           0.0   \n",
       "4                                  0.0                           0.0   \n",
       "...                                ...                           ...   \n",
       "1754150                            1.0                           0.0   \n",
       "1754151                            1.0                           0.0   \n",
       "1754152                            1.0                           0.0   \n",
       "1754153                            1.0                           0.0   \n",
       "1754154                            1.0                           0.0   \n",
       "\n",
       "         TERMINAL_ID_NB_TX_7DAY_WINDOW  TERMINAL_ID_RISK_7DAY_WINDOW  \\\n",
       "0                                  0.0                           0.0   \n",
       "1                                  0.0                           0.0   \n",
       "2                                  0.0                           0.0   \n",
       "3                                  0.0                           0.0   \n",
       "4                                  0.0                           0.0   \n",
       "...                                ...                           ...   \n",
       "1754150                            4.0                           0.0   \n",
       "1754151                            9.0                           0.0   \n",
       "1754152                            5.0                           0.0   \n",
       "1754153                            6.0                           0.0   \n",
       "1754154                           12.0                           0.0   \n",
       "\n",
       "         TERMINAL_ID_NB_TX_30DAY_WINDOW  TERMINAL_ID_RISK_30DAY_WINDOW  \n",
       "0                                   0.0                        0.00000  \n",
       "1                                   0.0                        0.00000  \n",
       "2                                   0.0                        0.00000  \n",
       "3                                   0.0                        0.00000  \n",
       "4                                   0.0                        0.00000  \n",
       "...                                 ...                            ...  \n",
       "1754150                            28.0                        0.00000  \n",
       "1754151                            39.0                        0.00000  \n",
       "1754152                            33.0                        0.00000  \n",
       "1754153                            28.0                        0.00000  \n",
       "1754154                            41.0                        0.02439  \n",
       "\n",
       "[1754155 rows x 23 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving of dataset\n",
    "\n",
    "Let us finally save the dataset, split into daily batches, using the pickle format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "DIR_OUTPUT = \"./simulated-data-transformed/\"\n",
    "\n",
    "if not os.path.exists(DIR_OUTPUT):\n",
    "    os.makedirs(DIR_OUTPUT)\n",
    "\n",
    "start_date = datetime.datetime.strptime(\"2018-04-01\", \"%Y-%m-%d\")\n",
    "\n",
    "for day in range(transactions_df.TX_TIME_DAYS.max()+1):\n",
    "    \n",
    "    transactions_day = transactions_df[transactions_df.TX_TIME_DAYS==day].sort_values('TX_TIME_SECONDS')\n",
    "    \n",
    "    date = start_date + datetime.timedelta(days=day)\n",
    "    filename_output = date.strftime(\"%Y-%m-%d\")+'.pkl'\n",
    "    \n",
    "    # Protocol=4 required for Google Colab\n",
    "    transactions_day.to_pickle(DIR_OUTPUT+filename_output, protocol=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The generated dataset is also available from Github at `https://github.com/Fraud-Detection-Handbook/simulated-data-transformed`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
